A preferred way of viewing video is by digital means. Among the benefits of digital viewing is the ability to quickly move to early and late in a given video stream, for example among chapters in a theatrical movie. To support this function, a ‘video scrubber’ is often displayed, generally as a thin horizontal strip under the frame of the video. Moving a cursor or similar control left and right determines the location in the video being displayed, with the extreme left being the beginning of the video and the extreme right being the end.
Using such a scrubber, a viewer can easily move backward and forward in the video. Similar methods are used for any stream, for instance, scrubbing audio. The method is applied in other domains, for example in reviewing sensor steams from military sensors or those use to control factories.
In some instances, the scrubber displays additional information. For example, some displays for video streaming services will indicate on the scrubber how much of the video has been downloaded from a central store to the viewing device. The user then is advised that the scrubber will be non-functional past that point. Because a primary purpose of scrubbers in the current art is navigation, some versions will display a preview frame of the video at that location as the cursor is moved left and right.
A weakness of current methods is that a viewer cannot anticipate what is where, and actually has to move back and forth to discover any content. That is, you discover ‘where you can go’ using the same control that takes you there. Another weakness of current methods is that it is not possible to make indications of where one would want to go. For example, theatrical movies are broken into scenes; but a viewer may want to go to a certain scene and the scrubber should provide some easy way to locate that scene.
This weakness in current methods extends to other guiding marks that a user might want to use. For example, a user might want to make notes about some object or event on the video. These notes might be simple, for example a text note, or rich as in the case of a comprehensive illustrated essay that points to a feature in a scene or frame. And they might also want to add information overlaid on the film itself; an example might be where the camera is placed and moves in relationship to the blocking of the scene and the set. This new system and method allows all of these new features.
In the general case, whether dealing with fictional or documentary video or other streams, there is a need for a dynamic scrubber that provides information about content before and after the current location without visiting it; provides an easy way to locate and browse to added information or markers for same; and supports the creation of rich on-screen graphical annotations.
A related need concerns the display of object and human paths over time, and/or movement of the environment. This latter may involve architectural features such as walls, which can progressively move as the camera pans, but can also address off-screen items such as the camera and light sources, the position of which might also need to be tracked. If these needs were addressed adequately, a vocabulary of cinematic effects can be modeled, annotated and browsed. This new system and method enables this.
An unrelated need concerns dynamic update. In the example of annotated theatrical films, the film itself is an unchanging artifact. But the annotations may be dynamically changing as collaborators add items and intelligent or human composition systems change the result. A more general case is that the base streaming file is dynamic, either because it is being modified, or because it is a continuous stream. Therefore, there is an additional need for a scrubber that indexes and allows browsing of both streaming media and annotations of various kinds that is also dynamically updatable.
In the current art, there is no computing system that can scan a video, collection of videos or other streams and extract features that can be used for advanced spatio-temporal navigation. Similarly, there is no computing system that manages dynamic ontologically-registered knowledge of a film to support advanced spatio-temporal navigation. There is, in the current art, no computing system that can combine the two and generate a display to be delivered for example over the internet to a browser to support advanced spatio-temporal navigation.
Therefore, there is a need for a computing system that can be modularized into components, one of which processes video streams to identify features for knowledge anchoring. A second cooperating computing system is needed to manage collected, consolidated information relevant to the video and deliver it in a fashion optimized for linking to the anchors determined by the first system. There is a need in the art for a third computing system to assemble the results of these two systems and deliver a navigable visual presentation to a consuming client computing system, for example a computing system that supports a web browser.